Dynamic Collections in Indri

Text search engines have historically been designed for unchanging collections of documents. While this is fine for many applications, a growing number of important applications in news, finance, law and desktop search require indexes that can be efficiently updated. Previous research into supporting dynamic collections revolves around incremental methods. Incremental systems are optimized for adding large batches of documents to an existing index. These systems do not generally allow for queries to run while an incremental update is taking place. This work presents recent changes to the Indri search engine to support dynamic collections. Unlike previous incremental systems, Indri does not require large batch sizes to achieve efficient indexing performance. Indri is also designed to be as concurrent as possible, allowing queries to run while documents are added to the system.